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This  paper  introduces  a  probabilistic  model  that  is  capable 
of  diagnosing  and  classifying  cognitive  errors  in  a  general 
problem-solving  domain.  The  model  is  different  from  the  usual 
deterministic  strategies  common  in  the  area  of  artificial 
intelligence  because  the  item  response  theory  is  utilized 
for  handling  the  variability  of  response  errors.  As  for 
illustrating  the  model,  the  dataset  obtained  form  a  38-item 
fraction  addition  test  is  used,  and  the  students'  responses 
are  classified  into  34  groups  of  misconceptions.  These  groups 
are  predetermined  by  the  result  of  an  error  analysis  previously 
done,  and  validated  with  the  error  diagnostic  program  written 
by  a  typical  formal  logic  approach.  -  . 
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Several  deterministic  strategies  commonly  used  in  the  area  of 
Artificial  Inttlligenc*  (AI)  havt  racantly  penetrated  into  tha  tatting  of 
procadural  doaaint  and  hava  tuccattfully  diagnosed  a  variety  of  cognitive 
errors  influencing  a  student's  performance  on  a  test.  Such  error 
diagnostic  systems  (Brown  &  Burton,  1978;  Tatsuoka,  Baillie  and 
Vaaaaoto,  1983;  Baillie  &  Tatsuoka,  1984)  are  able  to  provide  detailed 
descriptions  of  erroneous  rules  of  operation  resulting  from  tone  sources 
of  Misconceptions  or  incomplete  knowledge.  Such  a  diagnosis  of  student 
perforeance  on  a  test  provides  more  valuable  and  detailed  information  than 
the  total  score  of  the  test  can  do,  and  moreover  it  enables  us  to  optimize 
remediation  of  errors  and  to  evaluate  instruction  and  teaching  methods. 

However,  the  total  numbers  of  erroneous  rules  discovered  by  various 
cognitive-error-diagnostic  systems  are  quite  largei  88  in  signed-number 
addition  and  subtraction,  70  in  fraction  addition  and  104  in  whole  number 
subtraction  problems  (VanLehn,  1983).  A  single  misconception  often 
produces  several  different  erroneous  rules.  For  example,  when  adding  two 
fractions  with  different  denominators,  many  students  add  the  numerators 
and  denominators  separately  without  obtaining  the  least  common  denominator 
of  the  two  numbers.  Some  students  take  the  larger  denominator  instead  of 
the  common  denominator  and  add  the  numerators.  As  listed  in  Tatsuoka 
(1984a),  there  are  eleven  different  erroneous  rules  which  result  from  a 
misconception  originating  with  obtaining  the  least  common  denominator  of 
two  fractions  with  unlike  denominators. 

Tatsuoka  and  Tatsuoka  (1982)  introduced  an  index  to  measure  the 
degree  of  consistency  with  which  a  student  applies  his/her  own  rules 
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throughout  a  test.  The  index  (individual  consistency  index  or  I C I )  was 
applied  to  the  domain  of  signed-number  subtraction  problems.  A  summary 
list  of  161  students'  IC1  values  was  shown  in  Tatsuoka  and  Tatsuoka  (1983, 
pp.  222).  The  results  indicated  that  quite  a  few  of  the  students 
performed  irregularly  on  the  parallel  forms  of  subtests.  Deterministic 
strategies  cannot  handle  the  variability  of  response  errors  efficiently, 
and  hence  a  stochastic  approach  is  needed. 

The  purpose  of  this  study  is  to  introduce  such  a  probabilistic  model 
and  further  discuss  some  properties  of  the  model  (Tatsuoka,  1984b,  1985; 
Tatsuoka  t  Baillie,  1982). 

G e ometric  Represen t a tion  of  Binary  Response  Patt e r n s 

The  one-  or  two-parameter  logistic  model  is  used  in  this  study.  This 

section  begins  by  introducing  a  function  mapping  response  binary  vectors 

into  a  set  of  ordered  pairs.  Suppose  x  is  a  vector  of  scores  on  n  items, 

x  *  (x ! ,x2, . . . ,xn) ,  and  P(8)  is  a  vector  of  logistic  function  values  whose 

ith  component  is  given  by, 

Pj<e>  ■  - - - - 

1  +  expt-1 . 7a j  <8  -  bj)l 

where  b^  is  the  difficulty  of  item  j,  and  aj  is  the  discriminating  index. 

Let  T (8)  be  a  vector,  tT  (8)  , . . . ,T  (8) 3  where  T(9)  is  the  true  score,  or  the 

a? 

average  of  Pj<9)  over  the  n  items.  Then  the  cross  product  of  two  residuals, 
P j ( 9 )  -  T (8)  and  P j  < 8 )  -  xj  for  j»l,2,...,n  s  a  function  of  x  for 
a  given  9: 

n 

(1)  fA(x>  «  (P  (9 )  -  x,  P<9>  -  T  (8) )  ■  i  (P  * (9)  -  Xi)(Pi(9)  -  T  <8  >  >  . 
v  -v  „  .  ,  J  J  J 

J-l 


'-A  -1  . 


v  -.  .VA 
-•  -v-  - 


The  expectation  and  variance  of  ffl(x)  for  a  fixed  0  is  given  by 

"  i>V 


Equations  (2)  and  (3!  (Tatsuoka  &  Tatsuoka,  1983), 

(2)  E  <fe(xk>)  =  0 

xkie 

and 

n 

<3)  Var  <  f  ft  <  x  t  >  >  *  I  P  ^  )  UXeHPXd)  -  T  <e>  >  ^  . 

*kl0  9  “k  j  =  1  J  J 

where  xk  is  a  vector  whose  total  score  or  weighted  sun  of  aj  is  a 

sufficient  statistic  of  0  in  the  one-  or  two-par ameter  logistic  nodel , 


respectively.  If  ffl<x)  is  standardized,  it  becomes  the  standardized 

”  .v 

extended  caution  index,  <2,  introduced  in  Tatsuoka  (1984b). 

The  formula  of  the  standardized  extended  caution  index,  <2  is  given 


Since  the  total  score  is  a  sufficient  statistic  at  the  maximum  likelihood 

A 

estimate,  MLE  0,  in  the  one-parameter  logistic  model,  all  response  patterns 

A 

with  the  same  total  score  have  the  same  MLE  0.  For  example,  a  ten-item 
test  has  252  different  response  patterns  with  the  score  of  five.  These 
patterns  correspond  to  different  values  of  <2  (Tatsuoka,  1985) 

The  numerator  of  <2  is  divided  into  two  parts  in  Equation  (5);  one  is 
a  function  of  x  and  the  second  is  a  constant  value  when  0  is  fixed. 
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(5)  CP <e>  -  x,  P (8)  -  T ( 0 ) )  «  -<x,  PC©)-  T (8) ) + (P (8) ,  PC©)-  T (©) ) 

•V  W  iV  M  W  W  W  M  W  AT 

If  a  response  pattern  conforms  well  to  a  Guttnan  scale,  the  first 
tern  of  Equation  (5)  becomes  negative.  But  if  the  pattern  has  to  be 
reversed  to  form  a  Guttnan  scale,  then  -<x,  PC©)  -  T ( 0 ) >  Hill  be  positive. 

/V  (V  M 

The  tno  extreme  values  of  the  first  tern  of  Equation  (5)  are  obtained  by 
the  Guttnan  and  reversed  Guttnan  response  patterns,  respectively, 
f ^ C x >  correlates  highly  nith  likelihood  function,  L  *  *  Pj (8) * j (8) 1 

(Harnisch  !<  Tatsuoka,  1983),  and  the  value  of  fg(x>  reaches  the  largest  or 
smallest  as  the  likelihood  function,  L,  reaches  the  largest  of  smallest, 
respectively. 

If  we  replace  8  by  the  MXL,  8  in  Equation  (1),  then  f$(x)  is 

A  A 

uncorrelated  Hith  8  as  Hell  as  Hith  the  true  score  T C© )  (Tatsuoka,  1985). 

A  A 

Therefore,  the  napping  of  x  to  the  ordered  pair  of  0  and  or 

A  A 

to  the  ordered  pair  of  T C 8 >  and  produce  tno  orthogonal 

vector  spaces,  respectively.  Tatsuoka  (1983,  1985)  defined  this  vector 
space  as  "rule  space." 

The  distance  betneen  Xj  and  x2  for  a  given  8  in  the  rule  space  is 
given  by 

(6)  Dist  (Xj,X2>  *  Ifg(Xj)  -  fg(X2>l 

n 

-  «  I  Cx i  j  -  x2 j ) CP j (8)  -  T (8) )  I  . 
j*l 

Geometric  representation  of  erroneous  rules  of  operation 

The  responses  to  the  test  items  produced  by  erroneous  rules  in  a 
procedural  domain  are  scored  by  a  scoring  procedure,  nhich  may  either  be 


component  tearing  (Tatsuoka,  i 983 >  or  the  regular  right  or  wrong  scoring, 
and  art  thus  converted  into  a  set  of  biary  response  pattern  vectors.  The 
component  scoring  procedure  changes  the  unit  of  scoring  to  subcomponents 
of  procedural  steps.  For  example,  the  answers  of  signed-number  addition 
and  subtraction  problems  are  scored  separately  for  the  number  and  sign 
parts.  By  so  doing,  the  regular  response  patterns  are  decomposed  into  two 
sets  of  component  response  patterns.  Conversely,  the  elementwise 
multiplication  of  the  two  component  response  patterns  becomes  the  regular 
response  pattern.  Selection  of  appropriate  components  is  often  the  key  to 
representing  erroneous  rules  uniquely  by  a  set  of  binary  component 
response  patterns  (Tatsuoka  &  Tatsuoka,  1981}  Tatsuoka,  1983).  Similarly, 
the  answers  of  fraction  arithmetic  can  be  decomposed  into  three 
subcomponents,  the  whole  number,  the  numerator  and  the  denominator  parts. 

One  of  the  most  typical  erroneous  rules  observed  in  fraction  addition 
problems  is  to  add  the  correspond! ng  parts  separately.  For  the  problem, 

1  4/5  +1  3/7,  the  answer  based  on  this  rule  becomes  2  7/12.  But  for  another 
problem,  4/7  ♦  2/7,  the  answer  produced  by  the  rule  is  6/14,  whose 
numerator  happens  to  coincide  with  that  of  the  right  answer.  Therefore, 
the  component  response  pattern  vectors  of  the  two  problems  will  be  (1,1) 
for  the  whole  number  part,  (0,1)  for  the  numerator  part  and  (0,0)  for  the 
denominator  part.  The  elementwise  multiplication  yields  the  regular 
scores,  (0,0). 

More  generally,  suppose  x,  y,  and  z  are  such  component  response 

*¥  <V 

patterns  for  an  erroneous  rule,  then  they  will  be  mapped  into  the  three 
sets  of  ordered  pairs,  < ex ,  ?2x>,  <By,  ^y>  and  < ©z ,  <2z>  by  the  mapping 
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function  (1).  Therefore,  in  this  cate,  the  rule  it  represented  by  the 

AAA 

points  <ex ,  ?2x »  ®yi  ?2y’  S^z*  in  the  *ix_dimensional  space. 

However,  the  unit  o f  scoring  of  the  responses  to  the  test  items  does 
not  have  to  be  a  subcomponent  of  the  procedural  steps.  It  could  be  the 
regular  score  of  right  or  wrong,  depending  on  the  purpose  of  analysis. 

If  an  analysis  is  aimed  at  placing  a  new  student  into  his/her  most 
appropriate  achievement-level  group,  then  the  unit  of  analysis  may  be 
the  responses  to  the  test  items  without  any  finer  decomposition  into  the 
subprocesses.  For  example,  the  items  in  a  mathematics  placement  test 
can  be  decomposed  into  several  subtests  measuring  geometry,  elementary 
algebra,  advanced  algebra  or  trigonometry  < Or y ,  Mayberry  &  Yamamoto,  1965). 
Thus,  the  knowledge  measurable  from  a  subtest  could  be  the  analysis  of 
interest,  and  thus  replacing  an  erroneous  rule  in  a  procedural  domain  in 
the  analysis  of  cognitive  errors  by  several  levels  of  the  knowledge  in  an 
achievement  test. 

Illustration  of  Rule  Space  with  a  Fraction  Addition  Test 

All  erroneous  rules  of  operation  in  a  procedural  domain  can  be 

A 

represented  as  points  in  the  rule  space  along  with  the  response 

patterns  obtained  from  students'  performances  on  a  test.  Figure  1 

shows  the  rule  space  constructed  for  a  36-item  fraction  addition  test  given 


Insert  Figure  1  about  here 


to  595  students  in  a  local  junior  high  school,  with  four  erroneous  rules 
as  described  in  Shaw,  at  a  1 . ,  (1961).  Plus  marks  (  +  )  represent  real  data 
and  "o"'s  are  the  four  rules  listed  in  Table  1,  where  the  regular  scores 


Figure  1:  Plot  of  595  Students'  Scores  on  a  38-item  Fraction  Test  in 
the  Rule  Space 


are  used  to  calculate  these  points.  Although  only  four  items  are  shown  in 
Table  1,  the  points  correspondi ng  to  these  rules  in  Figure  1  are  obtained 


Insert  Table  1  about  here 


by  using  all  38  items  in  the  test.  As  can  be  seen  in  Figure  1,  the 
locations  of  Rules  13  and  17  are  identical,  while  those  of  Rules  11  and  10 
are  farther  apart  from  each  other.  Since  the  likelihood  of  rules 
correlates  highly  with  the  vlues  of  ?2i  the  rules  located  close  to  the 
0-axis  are  observed  more  often  than  those  located  in  an  upper  part  of  the 
space,  like  Rule  10.  The  response  patterns  by  the  regular  scoring  of 
Rules  13  and  17  are  identical,  and  hence  it  is  difficult  to  distinguish 
between  their  correspondi ng  points  in  the  rule  space.  The  number  of  the 
response  patterns  similar  to  those  of  Rules  13  and  17,  ones  for  the 
problems  of  like  denominators  and  zeros  for  unlike  denominators  turned 
out  to  be  large  (Tatsuoka,  Tatsuoka  &  Baillie,  1984),  and  the  number  of 
response  patterns  resulting  from  use  of  Rules  13  or  17  with  less 
consistency  for  the  items  simplifiable  before  the  addition  operation 
clustered  in  a  box  shape,  drawn  in  Figure  1.  However,  the  levels  of 
understanding  that  produce  Rules  13  and  17  are  quite  different.  The 
source  of  misconception  for  Rule  13  originates  from  obtaining  the  common 
denominators  of  the  two  like  fractions  while  Rule  17  originates  from 
gettinq  equivalent  fractions  after  the  correct  denominators  are 
obtained.  Therefore,  remediaton  of  the  error  types  relating  to  Rule  17 
should  be  easier  than  that  of  Rule  13.  In  order  to  distinguish  these 
different  levels  of  error  tvpes,  the  scores  for  denominators  must  be 
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levels  of  courses  in  Mathematics  more  efficiently  and  minimizes  the 
number  of  dropouts  and  maximizes  learning  of  a  new  topic  for  A-  or  B- 
students. 

However,  the  new  approach  requires  a  couple  of  technical  problems  to 
be  solved.  A  better  estimation  of  ellipses  will  enhance  the  analysis 
results.  A  better  parameter  estimation  technique  will  also  provide 
efficient  prescriptive  information  for  each  student.  An  accurate  decision 
of  classifying  each  point  to  one  of  ellipses  will  be  very  important. 

These  techniques  are  wide  open  for  further  research. 


possible  to  evaluate  the  ttacing  methods  and  redesign  thaa  for  more 
afficiant  inatruction. 

An  iaportant  charactariatic  of  tha  nodal  diffarant  from  other 
paychoaatric  techniques  auch  aa  cluater  analyaia,  multidimensional 
scaling,  and  factor  analyaia  ia  that  tha  nodal  can  control  claaaif ication 
of  parforaancaa  on  the  teat  by  aalacting  a  particular  aat  of  ellipaea. 
Thaaa  ellipaea  rapraaant  arronaoua  rulaa  in  a  procedural  donain,  or 
aourcaa  of  niaconceptiona  producing  a  variety  of  arronaoua  rulaa  of 
operation,  or  diffarant  achiavaaant  lavala.  Since  uaara  of  tha  nodal 
controla  the  input  of  ellipaea  according  to  their  need  or  intereat,  they 
can  avoid  the  tricky  quaation  about  interpretation  of  factora,  or 
cluatara.  Tha  quantity  expressed  by  tha  y-axis  repraaanta  tha  typicality 
of  a  reaponae  pattern  Mith  raapact  to  a  given  aanpla.  Ita  distance  fron 
tha  x-axis  (HIE  6  representing  state  of  knowledge)  provides  the 
infornation,  to  what  extant  a  particular  bug  is  likely  happen  in  tha 
aanpla.  This  probabliatic  interpretation  ia  a  unique  feature  of  tha  rule 
apace  nodal  which  enhances  an  already  useful  prescriptive  infornation 
about  teat  perfornancea  with  the  likelihood  of  such  an  incident. 

Since  the  nodal  is  a  generalized  version  of  aastery  testing  or 
criterion  testing,  it  is  applicable  to  the  conputer  aided  instruction  on 
ni cro-conputera  aa  an  integrated  part  of  training  prograna.  Diagnosing 
error  types  instead  of  using  the  total  score  wll  be  quite  useful  in 
training  students.  Moreover,  the  nodal  can  be  used  as  an  efficient 
placenent  tool  for  adni ni atrator a  of  training  prograna.  Ory,  at  al. 

(1905)  denonatrated  that  the  nodel  places  qualified  students  to  various 


Figure  J:  Some  Selected  Ellipses  from  Table  5 


Table  3 


Piraatttri  of  Fourtttn  Ellipati  with  tht  Numbtr  of  Studtnta  Cl ai»i i i ad 
in  Each  Ellipn  <N  »  S95) 


A 


Sat 

Fraquaociai 

Cintroid 

<1xA2 

1(1) 

4 

-0.21, 

-0.25 

11(3) 

1 

-0.35, 

3.41 

III (5) 

5 

0.28, 

0.66 

IV  (6) 

37 

0. 14, 

0.91 

V<tl) 

87 

0.66, 

0.75 

VI (12) 

12 

0.46, 

3.74 

VI I ( 13) 

184 

1.93, 

0.55 

VIII (17) 

18 

-0.31, 

-.33 

IX (22) 

11 

-0.04, 

0.43 

X  (23) 

23 

-0.22, 

-1.74 

XI (28) 

2 

0.  16, 

-1.00 

XII (29) 

13 

0.38, 

-0.51 

XIII (34) 

158 

-2.00, 

0.75 

Unclaaaifiad 


37 


(continued) 


I X ( 22  > 

Simplifying  error  with 
a  combination  of  addi¬ 
tion,  equivalent  frac¬ 
tion  and  converting 
errors. 

F+F  with  NS 
equivalently,  set 

5,  except  for 

S-type 

F  +  F  with  S 
all  mixed 

S  and  NS 

X  <  23) 

Si mpl i f yi ng  error  with 
combination  of  addition 
equivalent  fraction  and 
converting  errors. 

c*f  with  NS 
equivalently  set  6 
minus  S-type 

c*f  with  S 
all  cWf  types 

S  and  NS 

X 1(28) 

Simplifying  error  with 
combination  of  other 

errors 

Set  11  minus 

S-type 

Set  11  and  all 
F+F,  c*f  types 
with  S-types 

X I I (29) 

Simplifying  errors 

Set  15  minus  S-type 

Items  with  S-type 

XIII (34) 

Mostly  adding  corres¬ 
ponding  parts. 

A  few  items 

Almost  all  items 

F+F:  Two 

numbers  are  fractions  such  as  3  +  \ 

Mixed:  One  or  two  number*  are  mixed  such  a*  I7  +  or  ?  +  2; 

6  3  6  3 

c«f:  The  denominators  are  the  same  in  the  model  a^  +  d?  . 

c  f 

cFf:  The  denominators  are  different  in  the  model  a-  +  d£  . 

c  f 

Si  The  fraction  part(s)  can  be  simplified,  either  by  reducing  or 


converting  to  a  mixed  number  before  addition  is  carried  out. 


NS:  Fraction  part(s)  are  not  simplif iable. 


•1  Set  number  appeared  in  Tatsuoka,  Tatsuoka  &  Baillie  (1904). 


Table  2 


Fourteen  Sets _t h at  Represent;  Four teen  Different  Tyj> es  of 


Set 

Incomplete  Knowledge  or 
Representative  Bug 

Item  Type  Will  Get 
The  Right  Answer 

I  (i) 

Convert  mixed  numbers  to 
improper  fractions  by 
by  adding  three  parts  for 
the  numerator  and  use 

Rule  13  for  answers 

F+F  and  c*f,  both 

S  and  Ns 

11(3) 

Rule  10 

Mixed  and  c»f, 

S  and  NS 

III (5) 

Convert  mixed  numbers 
to  wrong  improper  frac¬ 
tions  and  use  righr  rule. 

all  F+F  types 

S  and  NS 

IV<6) 

If  c*f,  then  use  the  all  c»f  types 

right  rule.  If  cMf,  S  and  NS 

then  add  corresponding 
parts  separately  (Rule  13). 

If  c«f,  use  the  right  rule. 

If  c#f,  take  the  least  common 
denominator  and  add  the 
original  numerators  (Rule  17). 

V  <  1 1 ) 

Omit  the  whole  number 
after  using  the  right 
procedure  on  fraction 
parts. 

all  F+Ftypes 
all  c*f  types 

S  and  NS 

VI (12) 

all  c>f  types 
all  F+F  types 

S  and  NS 

VII (15) 

Use  the  right  rule 
consistently 

all  items  with 

S  and  NS 

VIII (17) 

Simplifying  error, 
converting  error  and 
addition  error 

F+F  and  c«f, 

NS 

In  other  words, 

Sot  i  except  for 
simplifiable  items 

Mi sconcepti qns 

Item  Type  Mill  Get 
The  Wrong  Answer 

Mixed  and  c"f 
both  S  and  NS 


all  F+F  types 
Mixed  and  c"f 
S  and  NS 

all  mixed  numbers, 
S  and  NS 


all  cWf  types 
S  and  NS 


Mixed  and  c*f 
S  and  NS 


mixed  and  c«f 
S  and  NS 


none 


F+F  and  c»f 
S 

mixed  and  cWf 
both  S  and  NS 


(continued) 
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Insert  Tables  2  &  3  about  here 


Insert  Figure  3  about  here 


The  sources  of  Misconception  or  the  incomplete,  partial  knowledge 
corresponding  to  each  set  are  given  in  the  second  coluan,  and  item  types 
producing  the  correct  answers  or  wrong  answers  by  the  set  are  listed  in 
the  third  and  fourth  columns,  respectively.  Table  3  summarizes  the 
centroids  of  the  14  ellipses  in  the  third  column  and  the  number  of 
students  diagnosed  as  having  the  error  type  associated  with  each  set. 
Figure  3  shows  14  ellipses  which  cover  the  area  in  the  rule  space  pretty 
well.  By  examining  which  points  are  found  in  particular  ellipses,  93. 8X 
of  students'  performances  on  the  38-item  test  are  diagnosed. 

OiscussiorL 

A  probablistic  model  that  is  capable  of  diagnosing  and  classifying 
cognitive  errors  is  introduced  in  this  paper.  The  model  enables  us  to 
deal  with  variability  of  response  errors  via  item  response  theory.  Since 
all  response  patterns  are  mapped  into  a  two  dimensional  vector  space 
spanned  by  MLE  9  and  which  is  defined  as  the  rule  space,  this 

approach  can  be  used  for  evaluating  different  teaching  methods  or  two 
expert  systems  constructed  by  different  models  of  a  single  problem  solving 
domain.  The  point  is  that  the  two  different  teaching  methods  will  produce 
significantly  different  numbers  of  students  committing  different  types  of 
bugs  <Tatsuoka  fc  Sharabash,  1985),  and  hence  they  produce  different 
clusters  in  the  space.  By  examining  these  clusters  closely,  it  is 


Hap  representation  of  misconceptions  in  the  rule  space 

it  it  shown  that  response  patterns  originating  from  a  singl*  source 
of  misconception  or  rule  with  a  few  slips,  cluster  in  the  vicinity  of  the 
point  representing  that  rule  in  the  rule  space  (Tatsuoka  l  Baillie,  1982) 
In  this  section,  several  sources  of  misconceptions  in  fraction  addition 
problems  identified  by  the  error  analysis  of  fraction  addition  (Shaw,  et 
i[. ,  1981;  Baillie  &  Tatsuoka,  1984)  will  be  mapped  into  various  regions 
of  the  rule  space.  Points  in  these  regions  follow  bivariate  normal 
distributions  (Tatsuoka  &  Tatsuoka,  1985)  so  that  they  can  be  expressed 
algebraically.  As  mentioned  earlier,  the  major  and  minor  axes  of  the 
ellipses  are  orthogonal  and  thus  the  equations  of  the  ellipses  are  given 
by 

(7)  (X  -  XR) '  £_1<X  -  XR)  *  const 

where  X  corresponds  to  a  student's  score,  XR  corresponds  to  the  responses 
to  the  items  generated  by  Rule  R,  and  the  covariance  matrix  £  is  a 
diagonal  matrix  with  the  variances  of  0  and  respectively.  (For  more 
information,  refer  to  Tatsuoka  I <  Tatsuoka,  1985). 

Fourteen  sets  representing  different  characteri sti cs  of  the 
performance  on  the  test  are  described  in  Table  2.  The  classif ication  of 
these  erroneous  rules  is  a  tricky  task  (Johnson,  Draper  fc  Soloway,  1982; 
Tatsuoka,  1984c).  Several  different  approaches  to  classifying  erroneous 
rules  in  fraction  addition  problems  are  tried,  but  in  this  study  the 
method  based  on  a  task  analysis  described  in  Birenbaum  and  Shaw  (1984) 
is  used  to  illustrate  how  our  rule  space  model  works. 


taken  into  account  because  tha  reaponae  pattern  of  each  component  given 
in  Table  1  ahowa  tha  danoainatora  produced  two  different  patterna. 

Figure  2  is  the  plot  of  ordered  paira  obtained  froe  the 
denominator-component  acorea  for  the  same  38-item  fraction  addition  teat 
uaed  in  Figure  1.  The  cluater  bounded  by  a  box  in  Figure  1  was  dispersed 
in  Figure  2,  and  the  two  points  associated  with  a  consistent  application 
of  Rules  13  and  17  resulted  in  two  distant  locations  as  shown  in  Figure  2. 


Insert  Figure  2  about  here 


As  for  Rule  10,  the  corresponding  point  falls  in  an  upper  part  of  the 
rule  space  because  use  of  Rule  10  is  not  popular.  Indeed,  it  is  very 
unusual  to  see  a  response  pattern  consisting  of  ones  for  harder  problees 
such  as  adding  two  mixed  numbers  and  zeros  for  the  simpler  addition  of  two 
fractions.  The  rule  space  enables  us  to  see  two  important  aspects  of  pure 
characteristics  attributed  to  each  erroneous  rule  resulting  from  sources 

A 

of  misconceptions.  The  first  element,  MLE  6  represents  the  state  of 
knowledge  where  a  spcecific  error  is  produced.  The  second  element 
expresses  the  extent  to  which  this  error  type  is  usual  and  popular  in  a 
given  population.  Points  located  closer  to  the  x-axis  are  usual  and 
frequently  observed  ones,  while  points  falling  in  the  upper  part  of  the 
space  are  unusual  and  less  frequent. 

A  natural  question  arises  as  to  whether  or  not  a  point  that  lies  very 
close  to  Rule  13  is  actually  yielded  by  Rule  13  with  a  few  slips  (or 
random  errors  resulting  from  no-perfectly  systematic  application  of  Rule 
13).  The  next  section  will  address  this  problem. 


Table  1 


F o u r  _E r o u b  Rules  of  Fraction  Addition  and  Their  Responses  to  Four  Items 

Rule  11  Rule  13 

Item  &  Answer  Response  R  D  N  W  Response  R  D  N  W 


3?  +  4^ 

* 

7  LL  0 

0 

1  1 

7y 

1111 

7  7 

14 

14 

■  8$ 

=  d 

7 

7 

i  t  i  = 

5 

1  o 

0 

0  1 

2 

0  0  0  1 

3  2 

6 

5 

5 

8  +  6  - 

14_ 

Li  o 

1 

1  0 

14 

1111 

5  5 

5 

10 

5 

■ 

4 

=  a 

«  2- 

5 

5 

5 

ii  +  LO 

» 

2LL  0 

0 

0  1 

2LL 

0  0  0  1 

2  7 

14 

9 

9 

.  213 

■  3- 

-  3  *  '5' 

14 

9 

T 

Rule 

17 

Rule 

10 

Item  it  i 

Answer 

Response  R 

I) 

N  N 

Response 

R  D  N  W 

3?  +  4^ 
7  7 

■  'H 

”4 

1 

1 

1  1 

'T 

l 

l 

1 

1 

■  4 

•  4 

i  ♦  1  = 

3  2 

5 

6 

2  *  1 

6  3 

0 

0 

0  1 

4 

0 

0 

0 

0 

8  +  6  « 
5  5 

Li  a  2* 

5  5 

14 

5 

1 

1 

1  1 

14 

5 

0 

1 

1 

0 

■ 

4 

’  4 

■4 

ii  ♦  to 
2  7 

,  iO. 

14 

0 

1 

0  0 

41 

l 

1 

1 

0 

Rule  11 

:  Adding 

correspond! ng 

part’s 

separ atel y 

Rule  13:  If  the  denomi nator s  are  different,  then  apply  Rule  11;  else  gets 
the  right  answer. 

Rule  17:  If  the  denominators  are  different,  then  get  the  common  denominator  and 
add  the  original  numerators;  else  gets  the  right  answer. 

Rule  10:  Append  a  one  to  all  fractions  and  apply  Rule  13. 


R  :  The  regular  score,  the  anwer  is  the  unit  of  scoring. 

D  :  The  denominator  of  the  answer  is  the  unit  of  scoring. 

N  :  The  numerator  of  the  answer  is  the  unit  of  scoring. 

W  :  The  whole  number  part  of  the  answer  is  the  unit  of  scoring. 
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